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other governing documents. Lastly, the Technology Metrics focal area concentrates on 
the importance of technology maturation and how to qualitatively and objectively assess 
technologies. Thorough research was conducted on these areas to demonstrate DoD’s 
recent efforts to bridge the gap and provide innovative solutions to the Warfighter within 
a reduced timeframe. 

Additionally, a case study analysis was conducted on a Defense Advanced 
Research Projects Agency (DARPA) Program, Boomerang, to highlight attributes of a 
successful technology transition. 
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I. 


INTRODUCTION 


A. BACKGROUND 

Technology transition has become an area of extensive focus within the 
Department of Defense (DoD) over the last several years. 


(DoD) is in the process of transforming the U.S. armed forces from a Cold 
War-era fighting force to one that is lighter, more flexible, and more 
reliant on technology. This fighting force will be able to respond to a wide 
range of asymmetric threats with speed and efficiency. Accelerating the 
transition of new technologies into defense systems will be crucial to 
achieving this military transformation. However, the typical time required 
for moving new materials and processing technologies from research to 
applications is at least 10 years, and many times even longer. Historical 
precedents for the transition of new technologies into defense systems 
have been neither fast nor efficient, (www.nap.edu, p. 1) 

To understand the motivation behind a shift in the manner DoD transitions 
technologies, one may simply examine the statistics of major systems that have faltered 
during development due to the inability to successfully accelerate the incorporation of 
advanced technology. 

Faster incorporation of new technologies into complex products and 
systems holds the possibility of ever-increasing advantages in cost, 
performance, durability, and new functionalities, (www.nap.edu, p. vii) 

More efficient and effective management of technology is critical to ensuring the 
Warfighter receives timely, innovative solutions within cost and schedule constraints 
while maintaining mission effectiveness. A Memorandum from the Under Secretary of 
Defense (Acquisition, Technology, and Logistics) (USD(AT&L)) to the Secretary of 
Defense, Subject: Top 5 Priorities for AT&L, August 5, 2002, clearly stated. 

Accelerating the flow of technology to the Warfighter is one of the top 
priorities of the [USD(AT&L)], as well as the Services, defense agencies, 
and other key defense organizations that help transition technology. (DoD- 
DAU, p. xv) 

As a result, new organizational structures, policies, and metrics have evolved. 
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The Valley of Death occurs around the Milestone B decision at the time the 
system is progressing from the Technology Development (TD) phase into the 
Engineering & Manufacturing (EMD) phase (as seen in Eigure 1). 


Technology Opportunities & Resources 



JCIDS-^ 


Acquisition Process->■ 


Eigure 1. Defense Acquisition Management System (Erom: Matthews, p. 40) 


Milestone B is the critical point in the Acquisition Management when the 
technology enters the Systems Acquisition Phase. At that time, 

The project shall exit the Technology Development Phase when an 
affordable program or increment of militarily useful capability has been 
identified; the technology and manufacturing processes for that program 
or increment have been assessed and demonstrated in a relevant 
environment; manufacturing risks have been identified; a system or 
increment can be developed for production within a short timeframe 
(normally less than 5 years for weapon systems); or, when the MDA 
decides to terminate the effort. During Technology Development, the user 
shall prepare the Capability Development Document (CDD) to support 
initiation of the acquisition program or evolutionary increment, refine the 
integrated architecture, and clarify how the program will lead to joint 
warfighting capability. The CDD builds on the ICD and provides the 
detailed operational performance parameters necessary to complete design 
of the proposed system. A Milestone B decision follows the completion of 
Technology Development. (AT&E, p. 19) 
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The period of time between TD and EMD also correlates with a Technology 
Readiness Level (TRL) 6, as defined by the Defense Acquisition University and shown in 
Figure 2, as “System/subsystem model or prototype demonstration in a relevant 
environment.” Historically, the S&T community has developed technologies through 
TRL 5, yet, acquisition programs focus on more mature technologies at the TRL 7. 


FRP 

MS 

C 

MS 

B 


MS 

A 


System Test, Flight 
and Operations 


9 Actual system “flight proven” through 
successful mission operations 

System/Subsystem 

Development 


8 Actual system completed and “flight qualified 
through test and demonstration 


7 System prototype demonstration in an 
operational environment 



Technology Demonstration 


6 System/subsystem model or prototype 
demonstration in a relevant environment 

Technology Development 


5 Component and/or breadboard validation in 
relevant environment 



4 Component and/or breadboard validation in 
laboratory environment 



Research to Prove Feasibility 


3 Analytical and experimental critical function 
and/or characteristic proof-of-concept 

Basic Technology Research 

2 Technology concept and/or application 
formulated 


1 Basic principles observed and reported 


Figure 2. Acquisition Cycle and Technology Readiness Levels 


B. DEFINITION 

A vital step in the authors’ research was to define technology transition. Contrary 
to the common wisdom, technology transition and technology transfer are significantly 
different. The Defense Acquisition University defines technology transition in the 
following manner: 

The process of applying critical technology in military systems to provide 
an effective weapons and support system—in the quantity and quality 
needed by the Warfighter to carry out assigned missions and at the “best 
value” as measured by the Warfighter. 
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The authors suggest that the concept of time be included in the definition. In 
today’s geopolitical instability and in the face of relentless adversaries, it is essential that 
the DoD performs technology transition in a practical yet expedient manner. Time is a 
critical factor in technology transition. 

On the other hand, 

Technology Transfer is the transfer or licensing of technology developed 
in Federal Laboratories to domestic partners, including industry, state and 
local governments, and academia. Key mechanism for the transfer are 
Cooperative Research and Development Agreements (CRADAs), Patent 
License Agreements, Educational Partnership Agreements, and DoD-wide 
Partnership Intermediaries, (acc.dau.mil, n.p.) 

Technology transfer, although a critical component of technology development, 
shall not be discussed in this document. 

C. CHALLENGES 

All programs and initiatives will face expected and unexpected challenges. 
Historical evidence shows DoD’s difficulties overcoming these challenges in 
transitioning technologies from S&T to EMD. 

DOD relies on its research laboratories and test facilities as well as 
industry and academia to develop new technologies and systems that 
improve and enhance military operations and ensure technological 
superiority over adversaries. Yet, historically, DOD has experienced 
problems in bringing technologies out of the lab environment and into real 
use. At times, technologies do not leave the lab because their potential has 
not been adequately demonstrated or recognized. In other cases, 
acquisition programs—which receive the bulk of DOD’s funding in 
research, development, testing and evaluation of technology—are simply 
unwilling to fund final stages of development of a promising technology, 
preferring to invest in other aspects of the program that are viewed as 
more vital to success. Other times, they choose to develop the 
technologies themselves, rather than rely on DOD labs to do so—a 
practice that brings cost and schedule risk since programs may well find 
themselves addressing problems related to technology immaturity that 
hamper other aspects of the acquisition process. And often, DOD’s 
budgeting process, which requires investments to be targeted at least 2 
years in advance of their activation, makes it difficult for DOD to seize 
opportunities to introduce technological advances into acquisition 
programs. In addition, it is challenging just to identify and pursue 


4 



technologies that could be used to enhance military operations given the 
very wide range of organizations inside and outside of DOD that are 
focused on technology development and the wide range of capabilities that 
DOD is interested in advancing. (GAO, p. 4) 

Along with the aforementioned challenges, three other contributory factors have 
constrained technology transition. 

• Lack of policy guiding and/or enforcing the implementation of technology 
transition processes 

• Lack of organizational initiatives and structure that encourage and foster a strong 
technology transition environment 

• Lack of a standardized metrics-based system to measure the maturity of a 
technology and the ability to correlate that information with the acquisition 
framework 

Addressing the challenges with a systematic approach has allowed for a gradual 
shift in mindset within DoD; the changes are being embraced. 

D. NOTIONAL CONSTRUCT 

Transitioning technology, or ’’bridging the gap” as organizations such as DARPA 
refer to it, is a complicated task requiring a tremendous cooperative effort amongst those 
shown in Figure 3. The combination of organizational transformation, policy 
adjustments, and a metric-based recognition of technology maturation provides a DoD a 
path forward to cross the dreaded valley of death. 
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Figure 3. Collaborative Influenees 


As indicated in Figure 3, there is a close-knit interdependence between the 
Warfighter, the Requirements Office, the Industry Partner, and the Stakeholder, which 
influences the resultant end-item solution. This tight interaction is critical throughout the 
technology transition phase to ensure the appropriate solution is delivered to the 
Warfighter. Unfortunately, this arrangement has historically delayed the rapid insertion 
of technology due predominantly to the requirements portion. DoD programs are highly 
requirements-driven programs; the requirements take lengthy periods of time to derive. 
Therefore, DoD made a strategic decision per the 2001 Quadrennial Defense Review in 
which the Joint Capabilities Integration and Development System (JCIDS) was stood up 
to replace DoD’s legacy Requirements Generation System (GAO, p. 4) 
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The JCIDS process requires that gaps in military capabilities be identified 
and potential materiel and nonmateriel solutions for filling those gaps be 
developed based on formal capability assessments. The results of these 
capability assessments are formally submitted as initial capabilities 
documents (ICD)—a capability proposal—by a military service, defense 
agency, COCOM, FCB, or other sponsor. ICDs are intended to document 
a specific capability gap or set of gaps that exist in joint warfighting 
functions and propose a prioritized list of various solutions to address the 
gap(s). (GAO, p. 5) 

The GAO study continues to state: 

The JCIDS process may lack the efficiency and agility needed to respond 
to warfighter needs—especially those that are near term—because the 
review and validation of capability proposals can take a significant amount 
of time. A proposal submitted to JCIDS can go through several review and 
comment resolution phases before consensus is reached on the proposal, 
and through several levels of approval before the JROC [Joint 
Requirements Oversight Council] validates the proposal. Our review of 
capability proposals submitted to JCIDS from fiscal years 2003 through 
2008 found that review and validation takes on average 8 to 10 months 
(see fig. 5). JCIDS and service officials also indicated that prior to 
submitting a JCIDS proposal, the sponsor can take a year or more to 
complete a capabilities-based assessment and get a proposal approved. In 
other words, 2 years or more can elapse from the time a capability need is 
identified by a sponsor to the time the capability is validated by the JROC. 

(GAO, p. 13) 

What if a requirement does not exist, but a technology or prototype developed in 
the S&T community has the potential to be of great benefit to the Warfighter? 
Unfortunately, in most if not all cases this transition will not occur due to the inability to 
acquire funding. This lack of appropriated funding to support non-JROC sanctioned 
efforts is disruptive to the technology development process and detrimental to mission 
effectiveness. An organizational shift to include funding set-asides and a bubble-up or 
technology push approach would more effectively address the immediate needs of the 
Warfighter. The men and women engaged in the daily activities of the battlespace can 
offer a tremendous amount of insight into what is needed. As opposed to waiting for 
vetted requirements to drive technology development, allow concepts and prototypes to 
bubble-up through the ranks, consequently utilizing the buy-in of the operator community 
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to address a capability gap with a specific technology. This improved organizational 
process would increase the probability of meeting or exceeding the performance 
expectations of the Warfighter and reduce the time necessary to hand-off the end-item 
solution. 

E. HISTORICAL BACKGROUND 

The attention given to technology transition is a fairly new initiative as compared 
to the lengthy history of DoD. Official legislation introducing the Technology Transition 
Initiative (TTI) was established by Congress in the FY2003 National Defense 
Authorization Act. TTI, enacted by the 107* Congress of the United States of America, 
was detailed as follows in H.R.4546: 

(a) Initiative Required - The Secretary of Defense, acting through the 
Under Secretary of Defense for Acquisition, Technology, and Logistics, 
shall carry out an initiative, to be known as the Technology Transition 
Initiative (hereinafter in this section referred to as the “Initiative”), to 
facilitate the rapid transition of new technologies from science and 
technology programs of the Department of Defense into acquisition 
programs of the Department for the production of such technologies. 


(b) Objectives - The objectives of the Initiative are as follows: 


(1) To accelerate the introduction of new technologies into operational 
capabilities for the armed forces. 

(2) To successfully demonstrate new technologies in relevant 
environment. (107th Congress, p. 37) 

Prior to the enactment of H.R.4546, limited activity occurred within DoD to 
improve upon the past performance of technology transition. One example of such an 
initiative is the Advanced Concept and Technology Demonstration (ACTD) Program. 

The Department of Defense's traditional approach to developing and 
building weapon and information systems has been criticized for taking 
too long, costing too much, and not adequately involving those who 
ultimately use the equipment. To address those problems, the Department 
of Defense (DoD) initiated the Advanced Concept Technology 
Demonstration (ACTD) program in 1994. (Joseph, n.p.) 
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The ACTD process, as seen in Figure 4, addresses the technology maturity and 
insertion issues abating technology transition from the TD Phase to the EMD Phase. 
However, process alone was not enough to ensure a successful transition. 


AC TD b«cef 


TVa JHii iial 
AcqiuidKiin 



Figure 4. ACTD Process (From: Joseph, n.p.). 


When DoD established the ACTD program in 1994, it did not include a 
formal approach to address any concerns that might arise about the 
transition stage of the projects....For ACTDs that began in and after 1996, 
the AT Office (Office of the Deputy Under Secretary of Defense for 
Advanced Technology) made one important change by requiring project 
managers to establish transition integrated product teams (IPTs) where 
applicable. Transition IPTs examine whether a program will be affordable, 
what the requirements for operating and supporting the residual capability 
will be, what strategies are needed for testing and evaluation, and whether 
there is adequate documentation for maintenance, training, or moving a 
system into procurement. (Joseph, n.p.) 

The ACTD Program has gradually transitioned to the Joint Concept and 
Technology Demonstration Program or JCTD; it “was first introduced into the 
President’s Budget in 2006” (Peterson, p. 9). 
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The new Joint Capability Technology Demonstration Program (JCTD) 
furthers this [the ACTD] concept by developing and maturing 
technologies to support the unique needs of the joint community in an 
even more adaptive and responsive process. (Krieg, p. 7) 

The JCTD program falls within the structure of the Rapid Fielding Directorate, 
OSD. The Rapid Fielding mission is, “Rapidly transition innovative concepts into critical 
capabilities that counter unconventional and time-sensitive threats” 
(www.acq.osd.mil/rfd/, n.p.). 

Since the inception of the ACTD/JCTD process, several additional initiatives and 
organizations have emerged. (The authors address these in Chapter III.) Yet one 
prominent organization evolved well before ACTDs and other technology transition 
initiatives—the Defense Advanced Research Projects Agency (DARPA). 

DARPA was created in 1958 as the Advanced Research Projects Agency 
(ARPA). The political and defense communities recognized the need for a 
high-level defense organization to formulate and execute R&D projects 
that would expand the frontiers of technology beyond the immediate and 
specific requirements of the Military Services and their laboratories. 
(www.darpa.mil . History) 

Since DARPA’s inception, the organization has proven to be very successful 
maturing technologies and establishing technology transition strategies. A study 
conducted by the Potomac Institute for Policy Studies in 2001 presented the following 
regarding the success of DARPA’s technology transition initiatives. 

The principal finding of the study is that DARPA's transition performance 
has been excellent over the past forty years, inserting over 120 products or 
technologies into fielded systems (about 3 per year). During the past 
decade, the Agency's record has been even better, about 5 per year. 

Finally, where data was available, we calculated transition rates and found 
them to be at a level exceptionally high according to industry's standards. 
Considering DARPA's other missions and its responsibility to foster high- 
risk/high-payoff ideas, the Institute's team considers these statistics quite 
impressive. (Richardson, n.p.) 


10 



II. TECHNOLOGY TRANSITION POLICY 


A. INTRODUCTION 

This section discusses the technology transition policies within the DoD. The 
major polices governing technology transition within the DoD are found in DoD 
Directive 5000.01, DoD Instruction 5000.02, and in the Defense Acquisition Guidebook. 
The major policies related to technology transition are explored below. 

B. DOD DIRECTIVE 5000.01 

DoD Directive 5000.01 outlines the Defense Acquisition System. It is applicable 
to all acquisition programs within DoD. This Directive defines the Defense Acquisition 
System (DAS) as “the management process by which the Department of Defense 
provides effective, affordable, and timely systems to the users” (p. 4). It defines an 
Acquisition Program as a “directed, funded effort that provides a new, improved, or 
continuing materiel, weapon or information system, or service capability in response to 
an approved need” (p. 4). 

The purpose of the DAS is to: 

Manage the nation's investments in technologies, programs, and product 
support necessary to achieve the National Security Strategy and support 
the United States Armed Forces. The investment strategy of the 
Department of Defense shall be postured to support not only today's force, 
but also the next force, and future forces beyond that. (DoD Directive 
5000.01, p. 3) 

It further states that 

the primary objective of Defense acquisition is to acquire quality products 
that satisfy user needs with measurable improvements to mission 
capability and operational support, in a timely manner, and at a fair and 
reasonable price. (DoD Directive 5000.01, p. 3) 

In order to ensure the continued technological superiority of U.S. forces over our 
adversaries’ successful technology transition is a key component in the DAS. DoD 
Directive 5000.01 specifically addresses technology development and transition. 
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It requires that a program: 

Address user needs; maintain a broad-based program spanning all 
Defense-relevant sciences and technologies to anticipate future needs and 
those not being pursued by civil or commercial communities; preserve 
long-range research; and enable rapid, successful transition from the S&T 
base to useful military products. (DoD Directive 5000.01, pp. 9-10) 

C. DOD INSTRUCTION 5000.02 

DoD Instruction 5000.02 implements DoD Directive 5000.01 and lays out the 
operation of the DAS. It is applicable to all acquisition programs within DoD. The 
Defense Acquisition Management System prescribed in the DoD Instruction 5000.02 is 
illustrated in Figure 5. 
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Figure 5. Acquisition Cycle 


Concerning technology transition, the technology development phase of this 
management system is of key importance. During this phase, promising new 
technologies must be matured to a sufficient state allowing them to bridge the Valley of 
Death occurring around the Milestone B decision and be successfully transitioned into an 
acquisition program. 

The purpose the technology development phase is to: 

Reduce technology risk, determine and mature the appropriate set of 
technologies to be integrated into a full system, and to demonstrate CTEs 
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[Critical Technology Elements] on prototypes. Technology Development 
is a continuous technology discovery and development process reflecting 
close collaboration between the S&T community, the user, and the system 
developer. It is an iterative process designed to assess the viability of 
technologies while simultaneously refining user requirements. (DoD 
Instruction 5000.02, p. 16) 

In order to enter the technology development phase the Milestone Decision Authority 
(MDA) must approve the material solution and the Technology Development Strategy 
(TDS). The TDS must explain the proposed acquisition approach, either single step to 
full capability, or evolutionary, which is the preferred approach. When an evolutionary 
strategy is being used, the TDS must explain the increments that the material solution 
will be divided into given the maturity of the technology involved in the system. 

DoD Instruction 5000.02 further states: 

If an evolutionary strategy is used, the initial capability represents only 
partial fulfillment of the overall capability described in the ICD, and 
successive technology development efforts continue until all capabilities 
have been achieved. In an evolutionary acquisition, the identification and 
development of the technologies necessary for follow-on increments 
continue in parallel with the acquisition of preceding increments, allowing 
the mature technologies to more rapidly proceed into the Engineering and 
Manufacturing Development (EMD) Phase. Each increment of an 
evolutionary acquisition program that includes a Milestone A shall have a 
Milestone Decision Authority (MDA)-approved TDS. (p. 18) 

DoDI 5000.02 requires that technology risk be managed and mitigated in order to 
help ensure the program meets its cost and schedule goals. To this end, DoDI 5000.02 
requires routine objective assessments of technology maturity and risk. DoDI 5000.02 
states that 

Technology developed in S&T or procured from industry or other sources 
shall have been demonstrated in a relevant environment or, preferably, in 
an operational environment to be considered mature enough to use for 
product development, (p. 19). 

This corresponds to at least TRE 6. DoDI 5000.02 requires that Technology Readiness 
Assessments be conducted and if the technology is determined not to meet the minimum 
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maturity level then an alternative technology meeting the minimum maturity level and 
satisfying the users’ requirements must be used. If that is not possible, then modifying the 
users’ requirements is discussed with the user. 

The DoD Technology Readiness Assessment (TRA) Deskbook (2009) defines a 

TRA as 

a formal, systematic, metrics-based process and accompanying report that 
assesses the maturity of technologies called Critical Technology Elements 
(CTEs) to be used in systems, (p. 1-1). 

CTEs are defined as 

A technology element is “critical” if the system being acquired depends on 
this technology element to meet operational requirements (within 
acceptable cost and schedule limits) and if the technology element or its 
application is either new or novel or in an area that poses major 
technological risk during detailed design or demonstration. (DoD, 2009, p. 

1 - 1 ) 

In order to help clarify this definition, the DoD TRA Deskbook further states that 
“If the technology represents a major risk, it should be identified as a CTE so that the 
TRA will include technical information that can be used to mitigate the risk” (p. 1-1). 
TRAs are conducted by an independent review team (IRT) composed of the necessary 
subject matter experts (SMEs). DoD acquisition programs must prepare a TRA at 
Milestone B and C. In accordance with 10 USC 2366b on Major Defense Acquisition 
Programs (MDAPs) the MDA must certify that the CTEs are at least at TRE 6. The law 
allows the MDA to waive this certification; however, this is only in extraordinary 
circumstances. 

In summary, the DoDI 5000.02 requires that CTEs be at least at TRE 6 in order to 
be used in the system past Milestone B. So, before and during the technology 
demonstration phase technology elements identified as critical must be managed so that 
they are matured to at least TRE 6 or else they will not be usable in that increment of the 
program. These technologies could be slated for further maturation and potential 
insertion into a future increment of the program once they have reached TRE 6 or they 
could be shelved with no plan for future use. 
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D. DEFENSE ACQUISITION GUIDEBOOK 


The Defense Acquisition Guidebook (DAG) is meant to complement DoDD 
5000.01 and DoDI 5000.02 and provides additional guidance to the Acquisition 
Community on implementing these policies. Relative to technology transition the DAG 
provides additional information concerning the content of the TDS. The DAG specifies 
that the TDS should include a technology maturation section. It states, 

The Technology Maturation section of the TDS should include discussion 
of the identified preliminary critical technology elements (CTEs) and the 
respective strategies for attaining at least Technology Readiness Level 
(TRL) 6 prior to Milestone B. (p. 2.2.8) 

The goals for the maturation of each of the CTEs should be laid out including any 
reviews, tests, and/or demonstrations of how the CTEs are maturing. As previously 
discussed for CTEs, a TRL level of at least 6 must be reached prior to Milestone B. A 
TRL 6 indicates the CTE has been demonstrated in a relevant environment. Therefore, in 
order to make this assessment, the relevant environment must be defined in the context of 
the specific system being developed. 

The Defense Acquisition Guidebook states that the: 

Technology Development Strategy should identify the relevant 
environment including potential threats against the capabilities desired and 
the physical environment (altitude, temperature, vibration, electro¬ 
magnetic, etc...), within which the materiel solution needs to operate. The 
characteristics of the environment should also be related to testing efforts 
in the Test and Evaluation Strategy, (p. 2.2.8.1) 

DoDI 5000.02 requires competitive prototyping involving at least two competing 
teams prior to Milestone B approval. The DAG provides additional guidance to DoD 
acquisition professionals regarding competitive prototyping. The DAG states that the 
TDS “should include a description of the prototyping purpose and the prototyping 
strategy at the system and subsystem levels” (p. 2.2.8.2). 
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It further states, 


The prototyping strategy should be competitive and provide for prototypes 
of the system or, if a system prototype is not feasible, for prototypes of 
critical subsystems before Milestone (MS) B approval, (p. 2.2.8.2) 

The use of competitive prototyping at the system or subsystem level is a valuable tool to 
help mature technology, reduce risk, provide the government with a better understanding 
of the technology, and increase contractor motivation (DoD, 2010). 

DoDI 5000.02 requires Program Managers (PMs) to consider the use of Small 
Business Research Innovation (SBIR) program technologies and “...give favorable 
consideration to successful SBIR technologies” (p. 14). The DAG provides further 
implementing guidance stating that the PM should 

...prepare a Technology Development Strategy (TDS) that appropriately 
uses the SBIR program to develop needed technologies, includes the use 
of technologies developed under the SBIR program, and gives fair 
consideration to successful SBIR technologies, (p. 2.2.10.1) 

The PM must examine available SBIR technologies for inclusion in the system, 
either in the current increment or for possible inclusion in a subsequent increment and 
plan accordingly to develop those technologies to the appropriate maturation level and 
transition them into the program. 
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III. ORGANIZATION 


A. PROGRAMS 

In the post-Cold War era, a change was necessary to remain the most 
technologically superior military in the world. At that point, DoD began to implement 
organizational transformations to account for a leaner, rapid response military with a 
global perspective. The DoD Acquisition System, up to the early ‘90s, was prohibitive 
to rapid, cooperative technology development and insertion. In order to address this 
predicament, DoD established several organizations that utilized non-traditional, industry 
practices to abate the difficulties bringing technologies to a suitable maturation state. 
Albeit these organizations may not remedy the technology transition difficulties, their 
intent was a paradigm shift in DoD acquisition. The following chart describes a portion 
of the DoD organizational initiatives. 


Program 

Management 

Characteristics that 
Facilitate Transition 

Advanced Concept/Joint 
Capabilities Technology 
Demonstration (AC/JCTD) 

Program direction and 
oversight by OSD 

Projects managed and 
executed by Military 
Departments, Defense 
Agencies and SOCOM 

Focuses on Joint Warfighter 
Functional Capabilities 
which are often under¬ 
resourced and with no one 
organization tasked to meet 
these needs 

Allows warfighters to 
evaluate a technology for its 
potential 

Agile Integration and 
Development 

Army 

Funds acceleration for 
selected high-payoff 
emerging 
technologies 

Improves technology 
readiness by accelerating 
the tech development 
schedule and/or performing 
detailed safety & validation 
tests in field/operational 
environment 
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Collaborative 

Technology Alliances 

Army 

Collaboration among 
Government-Industry- 
University researchers to 
achieve affordable transition 
of innovative technologies 

Defense Acquisition 
Challenge Program 
(DAC) 

OSD oversight 

Military Department/ 
Defense Agency 
management and execution 

Provides an “on-ramp” for 
industry and government to 
propose innovative new 
technology and equipment 
solutions for acquisition 
programs 

Foreign Comparative 

Testing (FCT) 

OSD oversight 

Military Department/ 
Defense Agency 
management and execution 

Helps find developed 
technologies in allied 
nations and funds their 
testing for potential 
procurement 

Future Naval 

Capabilities (FNC) 

Navy 

Involves near-term S&T 
efforts that deliver maturing 
technologies for more 
timely incorporation 

Rapid Deployment 

Capability (RDC) 

Navy 

Provides the ability to reach 
immediately to a newly 
discovered threat or to 
respond to significant and 
urgent safety 
situations 

Rapid Technology 

Transition Program 
(RTT) 

Navy 

Has a charter to rapidly 
transition technology from 
any source, including those 
not traditionally associated 
with defense technology, 
into PORs 

SBIR Commercialization 
Pilot Program (CPP) 

Army, Navy, Air Force 

OSD Oversight, Direction, 
and Coordination 

Identifies projects with 
potential to meet high 
priority military needs 

Provides assistance to affect 
rapid transition 

Technology Insertion for 
Savings Program (TIPS) 

Navy 

Concentrates on existing 
and/or COTS technology 
that can be formed quickly 
into the solution and has a 
high return on investment 
after insertion 
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Technology Transition 
Initiative (TTI) 

Program direction and 
oversight by OSD 

Projects managed and 
executed by Military 
Departments, Defense 
Agencies and SOCOM 

Accelerate the introduction 
of new technologies into 
operational capabilities for 
the armed forces 

Warfighter Rapid 

Acquisition Process 

Air Eorce 

Provides transition funding 
for development & fielding 
of successful experiments, 
demonstrations, & 
innovative approaches 

Complete, approved 
acquisition plan 

Budget Activity 7 


Table 1. Technology Transition Organizational Initiatives 


(Source: Department of Defense Report to Congress on Technology Transition, p. 12) 


All of these organizations foster a strong, open-communicative architecture, 
beginning with the user (the Warfighter) progressing through the S&T and EMD arenas. 
They were formed to fulfill a common goal—rapidly transition mature technologies to 
the Warfighter. Whether the organization is supplying the funding to bridge the S&T and 
EMD appropriation gap or providing over-arching management or actually developing 
and maturing the technology, they have impacted the Warfighter with highly functional, 
relevant solutions. Yet, much larger strides in organizational reform are needed to 
overcome DoD’s historically slow development cycle. Even after Mr. David Packer of 
the Blue Ribbon Commission on Defense Management lamented over the “unreasonably 
long acquisition life cycle of 10-15 years” (Packer, p. 47) in 1986 DoD still lacks the 
ability to reduce the acquisition cycle of a major system. This unfortunately 

leads to obsolete technology in our fielded equipment [and] we forfeit our 
five-year technological lead by the time it takes us to get our technology 
from the laboratory into the field. Packer, p. 47) 
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In addition to addressing a broad technology base, the various organizations and 
initiatives listed in Table 1 span across the joint services, a critical attribute in the 
modern-day military environment where a large portion of programs are jointly 
represented. This joint approach is intended to leverage the knowledge, skills, and 
abilities across the services and maintain commonality to more rapidly mature a 
technology. Solutions may have their slight differences between the services; however, 
they have a common goal to transition useful technologies such as the F-35 Joint Strike 
Fighter (JSF) Program. The JSF Program, originally referred to as the Joint Advanced 
Strike Technology (JAST) Program, was a joint effort between the Navy, Air Force, 
Marines, and the UK. 

The goal of the JAST program was not to have developed a new aircraft, 
but instead it was to mature the technologies that a new series of tactical 
aircraft could use. (www.jsf.mil, History/JAST) 


Tech Link 


DACP 

9% 


DPA Title I 
4% 


FC 

11 '? 



JCTD 

12% 


ACTD 

51% 


Figure 6. Technology Transition Initiative Funding Break-out 

(Source: Payton, Slide 9) 
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Program 

$M 

Advanced Concept 
Technology 
Demonstration (ACTD) 
(BA-3) 

$164 

Joint Capability 
Technology 
Demonstration (JCTD) 
(BA-3/4/5 & Procure) 

$40 

Joint V\^rfighting 
Program (JWP)-BA-3 

$10 

Foreign Comparative 
Testing (FCT)-BA-6 

$36 

Defense Production 
Act(DPA) TilJe III 

$13 

Tech nology T ransition 
Initiative (TTI)-BA-3 

$30 

TechLink(BA-3) 

$3 

Defense Acquisition 
Challenge Pgm 
(DACP)-BA-5 

$29 


Table 2. Technology Transition Initiative Funding ($) 
(Source: Payton, Slide 9) 


As demonstrated in Figure 1 and Table 2, a sufficient amount of funding has been 
injected into these new programs. Based upon these amounts and the number of new 
initiatives, it is clear the need for a more efficient technology transition process has been 
recognized within DoD. DoD must maintain vigilant oversight to ensure efforts that all 
within these and forthcoming programs do not become victims of traditional 10-15 year 
programs. 

B. COOPERATIVE INITIATIVES 

Reform of the acquisition process is now a driving force in the Department 
of Defense. A number of specific acquisition reform initiatives have been 
conceived, some borrowed from industry, and briefed at the highest levels 
of Government. Many have been mandated by the Secretary of Defense, 

Dr. William Perry, implemented by the Under Secretary of Defense for 
Acquisition and Technology, Dr. Paul Kaminski, the Services, and some 

21 






















have been enacted by Congress. One such initiative borrowed from 
industry that will fundamentally change the way the Department does 
business is Integrated Product and Process Development (IPPD). (Burt, 
Foreword) 

The IPPD process, initiated in the early 1990s 

is the DoD management technique that simultaneously integrates all 
essential acquisition activities through the use of multidisciplinary teams 
to optimize design, manufacturing, and supportability processes. One of 
the key IPPD tenets is multidisciplinary teamwork through Integrated 
Product Teams (IPTs). (acc.dau.mil) 

IPTs have become a necessity in the development of technologies, and are an 
essential mechanism to ensure the right technology gets to the right person at the right 
time. Subject matter experts from all of the acquisition disciplines are brought together 
in a cohesive team to develop a technology from concept through sustainment. This is 
contrary to the traditional stovepipe development processes historically followed within 
DoD. IPTs are extremely user-oriented in how they approach development. Users are 
empowered to work directly with scientists, engineers, manufacturing specialists, 
logisticians, etc. to “shape” a solution that most appropriately balances the requirements 
with the “wants” and “like to haves.” This constant, open communication and interaction 
is of great benefit as a technology matures and is eventually inserted into a system 
solution. It has been demonstrated on several large acquisition programs that well- 
managed IPTs are extremely successful transitioning technologies through the TRLs and 
MRLs and their associated milestones. 

C. INDUSTRY PROCEDURES 

The business practices of industry are a great asset to DoD and provide valuable 
lessons learned. Although industry does not follow the stringent acquisition regulations 
that DoD is required to adhere to and their predominant focus is profit, the manner by 
which they have transitioned technology has proven to be very successful. According to 
a GAO study, industry leaders focus on three key techniques: 
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• Strategic planning at the corporate level 

• A gated management review process that ensures a technology’s 
relevancy, feasibility, and transition readiness 

• Effective tools to solidify commitment, address transition issues, 
and gauge project progress and process effectiveness. (GAO, p. 9) 

To expand upon these techniques: 


Successful transition in leading companies starts with strong strategic 
planning followed by a structured technology development process led by 
research labs and supported by tools that pave the way for a smooth 
handoff to the product line. Strategic planning is considered a precursor to 
transition and allows managers to identify market needs so the company 
can quickly adapt its technology portfolios to meet those needs. A gated 
technology development process continually tests for relevancy and 
feasibility of technologies and gauges the commitment of product line 
managers to accept them. Once a technology is ready to transition, 
management and funding responsibilities gradually shift from the lab to 
the product line. By the end of transition, but before product development 
starts, the technology is validated as mature enough for use in the intended 
product. (GAO, p. 3) 


TECHNOUXSY DEVELOPyENT GATES 

Explore 

Technology Ideas and concepts 
are being explored 

A 

Develop 

Technology development activities 
are underway 

’lew Rfft 

k A 

Technology transition 
Technology Is ready to transition 
from lab to product line team 

iew 

k 

Deliverables 

• Technology is consistent with overall 
business strategy 

• Technology is promising and Is ikely to 
meet needs for potential product lines 

• Lab Identmes potential products where 
technology can be used 

• Key cost, benefit nsk. marketing, 
manufacturing, and He cycle manage¬ 
ment issues are Identified 

• Scalabllty approaches are Identified 

• Technologies considered to be 

Intellectual property are Identified 

Deliverables 

• Technology is consistent with techno¬ 
logy strategy and other relevant 
strategies 

• Labs have high degree of confidence 
the technology will work 

« Product line team agrees that the 
technology will meet Its needs 

• Technical requirements are Identified 

• Cost, benefits, and risks are quantified 

• Scalability approach is selected 

• Strategies for addressing Intellectual 
property rights are selected 

Deliverables 

• Technology project complies wrth 
technology strategy 

• Technology Is sound 

• Technology meets product require¬ 
ments 

• Cost, benefit and nsks are well 
understood 

• Technology can be scaled to a 
magnitude appropriate for pracboal 
application 

• Piquet line team agrees technology Is 
ready 

• Intellectual property rights methods 
have been pursued 

• Technology Is demonstrated In an 
operational environment 

« Technical documentation Is ready to be 
gnen to product line team 


Figure 7. Technology Development Gates (From: GAO, p. 12) 
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Depicted in Figure 7, each phase of an industry development cycle is gated and 
successful transition into the subsequent phase is predicated upon the completion of the 
deliverables. This strategy builds upon the concept of TRLs and MRLs as metrics 
throughout the acquisition life cycle. Both provide the PM with objective criteria to 
determine the maturity of a technology. 

Relationship managers are another key industry technique that has rarely been 
implemented within DoD. These managers are a critical communication focal point 
throughout the entire product development life cycle. 

With the exception of DARPA, DOD does not use relationship managers 
in the same manner as leading private companies. According to an Air 
Force lab official, relationship managers market technologies being 
developed by the labs or gather data about ongoing projects for senior lab 
management. Most communication about technology transition in DOD 
takes place through integrated product teams or during annual reviews of 
technology projects by the senior-level oversight boards for each of the 
services. Use of relationship managers for these purposes are helpful, but 
the managers do not necessarily serve as points of contact within the labs 
and acquisition communities, do not devote time toward efficiently 
transitioning technologies to multiple weapon system programs, and do 
not help identify and address systemic transition problems. 

Within DARPA, senior officers, called operational liaisons, focus on 
marketing and transitioning DARPA-sponsored technologies. According 
to the DARPA director [Dr. Anthony Tether], the liaisons have been very 
helpful with transitioning technologies because they are well practiced at 
using the command chain of their respective services and finding the right 
service contact at the right time. The liaisons: 

• Provide operational advice for planning and strategy development 

• Provide an understanding of service perspectives, issues and needs 
so that potential customers can be identified and effective 
agreements can be written 

• Draft and coordinate agreements between DARPA and the services 

• Direct technology insertion in the services. (GAO, pp. 32-33) 
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As a direct link to the Case Study in Chapter V, 


The DARPA [prior] director [Dr. Anthony Tether] credits operational 
liaisons for the quick transition of the Boomerang, an acoustic shot- 
detection system, from the lab to troops in Iraq. DARPA developed the 
system in response to feedback from Iraq that convoys were being 
engaged by snipers yet remained unaware of sniper attacks until a 
windshield was broken, a soldier was hit, or a vehicle was visibly 
damaged upon inspection at the end of the convoy mission. Within 60 
days of an urgent Army request, DARPA fielded the first Boomerang 
system. But DARPA’s director said the system did not hold up well in the 
extreme weather conditions and under wartime conditions. As Boomerang 
II was being prepared for fielding, the director said the operational liaisons 
helped craft a more realistic concept of operations, training package, and 
logistical support package to ensure that Boomerang II not only was 
technologically ready for combat but was properly supported with spare 
parts, maintenance facilities, maintenance personnel, training, and lessons- 
leamed feedback to DARPA and the Army. The liaisons also ensured that 
the Army acquisition community was alerted to Boomerang IPs 
deployment so product developers would be ready to evaluate the final 
product for movement into the more traditional acquisition process. 
(GAO, p. 33) 


An organizational transformation is slowly infiltrating DoD, yet traditional 
roadblocks are inhibiting a more rapid progression. DoD’s lack of success is made 
evident in Table 3: 
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Table 1: Cost and Schedule Outcomes for 23 Programs Initiated under the Revised Policy (as of December 2005) 

Program 

Percent growth in estimated 
development cost* 

Percent growth in estimated 
development schedule 

Expeditona^ Fighting Vehicle 

61% 

70% 

Active Bectronically Scanned Array radar (upgrade 
tor F/A- 1 8 E/F fighter.'attack aircraft) 

14% 

1% 

Global l-lawk unmanned aerial vehicle 

166% 

Undetermined 

Joint Stnke Fighter 

30% 

23% 

UH.00M helicopter upgrade 

151% 

25% 

C-t 30 Avionics Modernization Program 

122% 

Undetermined 

C-S Reliabiity Enhancement and Re^engsting 

Program 

0% 

25% 

Joint Tactical Radio System Cluster i 

31% 

44% 

Joint Tactical Radio System Waveform 

44% 

Undetermined 

AdvarKod Anti-radiation GuKled Missie 

7% 

0% 

Mult-Platform Radar Technology Insertion Program 

0% 

Undetermined 

Future Combat System 

48% 

53% 

E-2 Advanced Hawkeye 

5% 

0% 

Warfighter Informaton Network-Tactical 

0% 

0% 

Small Diameter Bomb 

0% 

0% 

EA-t8G 

7% 

0% 

Joint Tactical Radio System Cluster 5 

0% 

2% 

Mult-Mission Maritime Aircraft 

0% 

0% 

Standard Missile-6 Extended Range Active Missile 

Block t 

0% 

0% 

Aenal Convnon Sensor 

45% 

36% 

B-2 Radar Modemizabon Program 

0% 

0% 

Patriot/Medium Extended Air Defense System 

Combined Aggregate Program (fire unit) 

0% 

0% 

Mission Banning System 

0% 

0% 


Table 3. DoD Major Programs Cost and Schedule Performance 

(Source: GAO, p. 36) 


The topics addressed above are a select few of the contributing factors to the 
future success of DoD’s technology transition abilities. Despite the numerous programs, 
initiatives, and lessons learned from industry, DoD’s traditional methods of conducting 
business are still prevalent throughout the acquisition cycle, in particular, the technology 
transition phase. Unfortunately until sound processes, procedures, and techniques are 
fully implemented and adhered to the Warfighter suffers the greatest impact. It is 
incumbent upon DoD to not only promote an organizational transformation with regards 
to technology transition but to take a step back and look at the big picture; absorb the 
positive attributes of the current structure and discard the negative attributes. The 
organizational mindset alone can be a powerful tool to bridge the gap between S&T and 
Engineering. 
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IV. TECHNOLOGY METRICS 


A. BACKGROUND 

As stated in the DoD’s report to Congress in July 2007, “Transitioning technology 
into established Programs of Record (PoR) is a longstanding challenge for the 
Department. The underlying problem has come into sharp focus in recent years with the 
adoption of Technology Readiness Levels (TRLs) as a common vocabulary for 
discussing the maturity of a technology and uncovering the disconnect between our S&T 
and acquisition communities. Our acquisition policies require a minimum of TRL 7 
("system prototype demonstrated in an operational environment") for a critical 
technology to be incorporated in a production program (an important best practice 
recommended by GAO to control technical risk, and strongly supported by DoD). On the 
other hand, expectations within S&T community have traditionally been to advance new 
technologies only to the TRL 5 level ("component and/or breadboard validation in a 
relevant environment" (e.g., high fidelity laboratory)), with no particular capability 
deployment in mind, and then move on to the next technology. Although a technology 
maturity gap exists, one now has the ability to exploit a metrics-based system to measure 
the breadth of that gap through Technology Readiness Levels (TRLs). Therefore, TRLs 
are not simply an assessment tool but also a planning and risk mitigation tool. 

The introduction of Technology Readiness Levels (TRLs) over the last 
decade provided an accepted common language and measurement scale 
intended to strengthen communication within and between the DoD S&T 
and acquisition communities, both in government and industry. 
Stakeholders in all parts of the acquisition system now expect that a 
Critical Technology Element (CTE) will have an appropriate TRE, or 
level of maturity, prior to acceptance as a baseline technology for a 
weapon system. They also embrace the concept that technology should 
have only an acceptable level of risk in order to pass through each 
acquisition milestone decision point and that TREs are a way of measuring 
and communicating that risk. (JDMTP, ES-5) 


27 



B. TECHNOLOGY READINESS LEVELS 


In 1991, the Office of Aeronautics and Space Technology (OAST) of the National 
Aeronautical and Space Administration (NASA) developed an Integrated Technology 
Plan (ITP). 

The purpose of the ITP was to serve as a strategic plan for the OAST 
space research and technology (R&T) program [...] conducting 
technology development that support future U.S. civil space missions. 
(NASA, ii) 

Within the ITP, nine definitive levels of technology maturity were defined. Those 
nine levels (shown in Table 4) have become the backbone of measuring technology 
development; although the definitions have evolved somewhat over the years (and across 
different organizations) the general premise remains the same. Technology development 
requires a metric to measure the maturity of the technology to ensure the appropriate 
level of effectiveness is obtained prior to transitioning to the next phase. Until NASA’s 


ITP, this uniform means of assessment did not exist. 


Technology Readiness Level 

Description 

1. Basic principles observed and reported. 

Lowest level of technology readiness. 
Scientific research begins to be translated 
into applied research and development. 
Examples might include paper studies of a 
technology’s basic properties. 

2. Technology concept and/or application 
formulated. 

Invention begins. Once basic principles are 
observed, practical applications can be 
invented. Applications are speculative and 
there may be no proof or detailed analysis to 
support the assumptions. Examples are 
limited to analytic studies. 

3. Analytical and experimental critical 
function and/or characteristic proof of 
concept. 

Active research and development is initiated, 
including analytical studies and laboratory 
studies to physically validate analytical 
predictions of separate elements of the 
technology. Examples include components 
that are not yet integrated or representative. 

4. Component and/or breadboard validation 
in laboratory environment. 

Basic technological components are 
integrated to establish that they will work 
together. This is relatively “low fidelity” 
compared to the eventual system. Examples 
include integration of “ad hoc” hardware in 
the laboratorv. 
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Technology Readiness Level 

Description 

5. Component and/or breadboard validation 
in relevant environment. 

Fidelity of breadboard technology increases 
significantly. The basic technological 
components are integrated with reasonably 
realistic supporting elements so it can be 
tested in a simulated environment. Examples 
include “high fidelity” laboratory integration 
of components. 

6. System/subsystem model or prototype 
demonstration in a relevant environment. 

Representative model or prototype system, 
which is well beyond that of TRL 5, is tested 
in a relevant environment. Represents a 
major step up in a technology’s 
demonstrated readiness. Examples include 
testing a prototype in a high-fidelity 
laboratory environment or in simulated 
operational environment. 

7. System prototype demonstration in an 
operational environment. 

Prototype near, or at, planned operational 
system. Represents a major step up from 
TRL 6, requiring demonstration of an actual 
system prototype in an operational 
environment such as an aircraft, vehicle, or 
space. Examples include testing the 
prototype in a test bed aircraft. 

8. Actual system completed and qualified 
through test and demonstration. 

Technology has been proven to work in its 
final form and under expected conditions. In 
almost all cases, this TRL represents the end 
of true system development. Examples 
include developmental test and evaluation of 
the system in its intended weapon system to 
determine if it meets design specifications. 

9. Actual system proven through successful 
mission operations. 

Actual application of the technology in its 
final form and under mission conditions, 
such as those encountered in operational test 
and evaluation. Examples include using the 
system under operational mission conditions. 


Table 4. Technology Readiness Levels 
(Source: DAU, Ask a Professor) 


Clarifying definitions (via DAU/Ask a Professor): 

• BREADBOARD: Integrated components that provide a 
representation of a system/subsystem and which can be used to determine 
concept feasibility and to develop technical data. Typically configured for 
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laboratory use to demonstrate the technical principles of immediate 
interest. May resemble final system/subsystem in function only. 

• HIGH FIDELITY: Addresses form, fit and function. High fidelity 
laboratory environment would involve testing with equipment that can 
simulate and validate all system specifications within a laboratory setting. 

• LOW LIDELITY: A representative of the component or system that 
has limited ability to provide anything but first order information about the 
end product. Low fidelity assessments are used to provide trend analysis. 

• MODEL: A reduced scale, functional form of a system, near or at 
operational specification. Models will be sufficiently hardened to allow 
demonstration of the technical and operational capabilities required of the 
final system. 

• OPERATIONAL ENVIRONMENT: Environment that addresses all 
of the operational requirements and specifications required of the final 
system to include platform/packaging. 

• PROTOTYPE: The first early representation of the system which 
offers the expected functionality and performance expected of the final 
implementation. Prototypes will be sufficiently hardened to allow 
demonstration of the technical and operational capabilities required of the 
final system. 

• RELEVANT ENVIRONMENT: Testing environment that simulates 
the key aspects of the operational environment. 

• SIMULATED OPERATIONAL ENVIRONMENTAL: Environment 
that can simulate all of the operational requirements and specifications 
required of the final system or a simulated environment that allows for 
testing of a virtual prototype to determine whether it meets the operational 
requirements and specifications of the final system. 


Technology Readiness Assessments 

The implementation within DoD of a TRL system to assess the maturity of a 
technology has more vividly exposed the gap between S&T and Engineering. As 
components/systems traverse through the various early development phases, the S&T, 
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Engineering and Test, and Stakeholder communities now have the appropriate 
technology metrics to prepare for the Valley of Death by exploiting the usage of 
Technology Readiness Assessments (TRAs). 

Per DoD Instruction 5000.2, the TRA is a regulatory information 
requirement for all acquisition programs. The TRA is a systematic, 
metrics-based process that assesses the maturity of Critical Technology 
Elements. The TRA should be conducted concurrently with other 
Technical Reviews, specifically the Alternative Systems Review, System 
Requirements Review, or the Production Readiness Review. If a platform 
or system depends on specific technologies to meet system operational 
threshold requirements in development, production, and operation, and if 
the technology or its application is either new or novel, then that 
technology is considered a Critical Technology Element. 

The TRA should not be considered a risk assessment, but it should be 
viewed as a tool for assessing program risk and the adequacy of 
technology maturation planning. The TRA scores the current readiness 
level of selected system elements, using defined Technology Readiness 
Eevels. The TRA highlights critical technologies and other potential 
technology risk areas that require program manager attention. 
(dag.dau.mil. Section 4.3.2.4) 

These analyses, along with a conducive organizational mindset, enhance the ability of the 
technology developers to plan for maturation to TRE 6 and ultimately TRE 7. 

The more information a PM is able to obtain about their technology’s 
developmental progression the better they can plan and develop technology transition 
agreements and establish the appropriate technology transition partners. 

C. MANUFACTURING READINESS LEVELS 

Another metric available to Program Managers are Manufacturing Readiness 
Levels (MRLs). 

Manufacturing Readiness is the ability to harness the manufacturing, 
production, quality assurance, and industrial functions to achieve an 
operational capability that satisfies mission needs—in the quantity and 
quality needed by the war-fighter to carry out assigned missions at the 
“best value” as measured by the Warfighter, (dag.dau.mil. Section 
4.4.14.2) 
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Throughout the development process, even during the early phases, appropriate 
production planning is critical to a successful transition. During TRAs MRLs are 
influential factors in the decision to transition a technology from the S&T environment 
into the EMD phase. Evaluators will ask, “Is the item producible? “Can the item be 
produced in a cost effective manner?” and “Are there current processes to produce the 
item?” These questions, and many others, help define the readiness of a technology and 
its ability to bridge the gap between TRE 5 and TRE 7. Even if one conceptualizes the 
most amazing design, if you cannot practically manufacture it then it serves no purpose to 
the Warfighter. 

There are ten MREs. These levels directly relate to the nine Technology 
Readiness Eevels that are in use with an additional MRE 10 that is equal to a program in 
full rate production. The first three levels are discussed as a single level, which is equal to 
TREs 1 through 3. Below are definitions of each MRE and a description of the criteria 
necessary to each level and the associated acquisition phase: 


MRL 1-3 

The organization has identified manufacturing concepts. This is the Pre- 
Concept Refinement phase. Identification of current manufacturing 
concepts or producibility needs has occurred and is based on laboratory 
studies. 

MRL 4 

The organization has validated the system, component, or item in 
laboratory environment. This is the lowest level of production readiness. 
The Concept Refinement (CR) phase leads to a Milestone A decision. 
Technologies must have matured to at least TRE 4. At this point few 
requirements have been validated and there are large numbers of 
engineering/design changes. The organization has not defined component 
physical and functional interfaces. Materials, machines and tooling have 
been demonstrated in a laboratory environment. Inspection and test 
equipment have been demonstrated in a laboratory environment. DTC & 
Production drivers identified. Producibility assessments have been initiated. 

MRLS 

The organization has validated component or item in an initial relevant 
environment. Engineering application/bread board, brass board 
development is occur-ring. This is the first half of the Technology 
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Development (TD) phase and merges with the second half when we begin 
system validation leading to a Milestone B decision. Technologies must 
have matured to at least TRL 5. At this point all requirements have not 
been validated and there are significant engineering/design changes. 
Industrial Base analysis has been accomplished to identify potential 
sources. Initial producibility of component technology has been completed. 
Form, Fit & Function constraints identified and allocated at component 
level. Key Performance Parameters allocated at component level and initial 
evaluation of Key Characteristics accomplished. Subsystem and major 
component level DTC goals established. Manufacturing cost considerations 
affect technology choices. Manufacturing cost drivers/goals identified. 
DTC/Production costs estimated and tracked. Required Manufacturing 
Technology (ManTech) efforts initiated. Yield/rate issues identified. Key 
Quality Characteristics identified. Science and Technology/ Special Test 
Equipment (ST/STE) requirements identified. Initial Manufacturing Plan is 
developed. 

MRL6 

The organization has validated the system in an initial relevant 
environment. Engineering application/bread board, brass board 
development is occurring. This is the 2nd half of the Technology 
Development (TD) phase and leads to a Milestone B decision. 
Technologies must have matured to at least TRE 6. All requirements have 
not been validated and there are significant engineering/design changes. 
Component physical and functional interfaces have not been defined. 
Materials, machines and tooling have been demonstrated in a relevant 
environment but most manufacturing processes are in development (e.g. 
ManTech initiatives). Inspection/test equipment has been demonstrated in a 
laboratory environment. Producibility assessments are ongoing initial 
improvements begun. Production cost drivers and goals are being analyzed 
and set. DTC goals have been set. 

MRL7 

The System, component or item is in advanced development. This is the 
System Development & Demonstration Phase (pre DRR). All technologies 
have matured to at least TRE 7. At this point engineering/design changes 
should be decreasing. Physical and functional interfaces should be clearly 
defined. All raw materials are in production and available to meet the 
planned ERIP schedule. Pilot line manufacturing processes have been set¬ 
up and are under test. Processes and procedures have been demonstrated in 
a production relevant environment. During this phase, the producibility 
improvements should be underway. DTC estimates are within 125% of the 
DTC goals. Production estimates are being established. 
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MRL8 

The system is in System Development & Demonstration leading to a 
Milestone C decision. Component or item is in advanced development and 
ready for low rate initial production. Technologies must have matured to at 
least TRL 8. Engineering/design changes should be decreasing 
significantly. There must be very few changes at the end of this phase. 
Physical and functional interfaces should be clearly defined. All raw 
materials are in production and are available to meet the planned LRIP 
schedule. Manufacturing processes and procedures have been proven on 
the pilot line, under control and ready for low rate initial production. 
During this phase, producibility risk assessments should be completed. The 
DTC goals should have been met. Production estimates meet production 
goals. 

MRL9 

The system, component or item has been previously produced or is in 
production. Or, the system, component or item is in low rate initial 
production. This phase is Low Rate Production & Deployment leading to a 
Full Rate Production Decision (FRP). During low rate initial production all 
systems engineering/design requirements should be met and there should 
be minimal system engineering/design changes. Technologies must have 
matured to at least TRL 9. All materials are in production and available to 
meet planned production schedules. All manufacturing processes are 
established and controlled in production to three-sigma or some other 
appropriate quality level. Machines, tooling and inspection and test 
equipment deliver three-sigma or some other appropriate quality level in 
production. Production risk monitoring is ongoing. LRIP costs meet 
production goals. 

MRLIO 

The system, component or item previously produced or in production. Or, 
the system, component or item is in full rate production. This is the Full 
Rate Production or Sustainment phase. This is the highest level of 
production readiness. There are minimal engineering/design changes. 
System, component or item is in production or has been produced and 
meets all engineering, performance, quality and reliability requirements. 
All materials, manufacturing processes and procedures, inspection and test 
equipment, controlled in production to six-sigma or some other appropriate 
quality level in production. A proven, affordable product able to meet 
required schedule. Production goals meet actual. (JDMTP, 16-18) 
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During the era of stovepipe development, manufacturability and producibility 
were afterthoughts in the development process. As DoD has learned over many years, 
this lack of insight into downstream processes is a recipe for disaster. Hence, the 
introduction of MRLs and Manufacturing Readiness Assessments (MRAs) into the pre- 
EMD phases of the development cycle. These assessments are critical elements in 
determining the true maturity of a technology and the potential big picture impact on the 
system. Similar to TRLs, MRLs provide a direct correlation to the acquisition life cycle. 
Additionally, MRLs are an excellent tool to assist in the transition between a prototype 
and production units, which is critical to attain the higher TRLs. 

D. SYSTEM READINESS LEVELS 

Technology and Manufacturing Readiness Levels focus on the maturity of 
components and sub-component whereas they do not fully assess the interoperability and 
interfacing of the components in a system. To encompass the overall maturity and ability 
of a system to transition a broader metric is warranted. The Stevens Institute of 
Technology has suggested the usage of System Readiness Levels (SRLs), which combine 
TRLs and Integration Readiness Levels (IRLs). 

The SRL Model is a function of the individual Technology Readiness 
Levels (TRL) in a system and their subsequent integration points with 
other technologies, the Integration Readiness Level (IRL). (Sauser, p. 10) 

The IRLs and SRLs are outlined in Table 5. 
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IRL 

Definition [9] 

7 

The integration of technologies has been verified and validated with sufficient 
detail to be actionable. 

6 

The integrating technologies can accept, translate, and structure information 
for its intended application. 

5 

There is sufficient control between technologies necessary to establish, 
manage, and terminate the integration. 

4 

There is sufficient detail in the quality and assurance of the integration between 
technologies. 

3 

There is compatibility (i.e. common language) between technologies to orderly 
and efficiently integrate and interact. 

2 

There is some level of specificity to characterize the interaction (i.e. ability to 
influence) between technologies through their interface. 

1 

An interface (i.e. physical connection) between technologies has been identified 
with sufficient detail to allow characterization of the relationship. 


Table 5. Integration Readiness Levels 
(Source: Sauser, p. 11) 
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SRL 

Name 

Definition 

5 

Operations & Support 

Execute a support program that meets 
operational support performance 
requirements and sustains the system in 
the most cost-effective manor over its 
total life cycle. 

4 

Production & Development 

Achieve operational capability that 
satisfies mission needs. 

3 

System Development & 
Demonstration 

Develop a system or increment of 
capability; reduce integration and 
manufacturing risk; ensure operational 
supportability; reduce logistics footprint; 
implement human systems integration; 
design for producibility; ensure 
affordability and protection of critical 
program information; and demonstrate 
system integration, interoperability, 
safety, and utility. 

2 

Technology Development 

Reduce technology risks and determine 
appropriate set of technologies to 
integrate into a full system. 

1 

Concept Refinement 

Refine initial concept. Develop 
system/technology development 
strategy 


Table 6. System Readiness Levels 
(Source: Sauser, p. 7) 


The SRL table (less SRL 5 which is beyond the scope of this document) and the 
IRL table outline critical factors in the maturation state of a system and provide more 
thorough indicators to the readiness of a system (or a group of technologies) to transition. 
However, these authors would argue the Stevens Institute neglected the inclusion of 
MRLs in their SRL model. As indicated previously, in Section C, MRLs are a vital 
assessment tool and would factor well into the SRL model. 
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V. DARPA CASE STUDIES 


A. BACKGROUND 

1. DARPA 

DARPA’s original mission, inspired by the Soviet Union beating the 
United States into space with Sputnik, was to prevent technological 
surprise. This mission has evolved over time. Today, DARPA’s mission is 
to prevent technological surprise for us and to create technological 
surprise for our adversaries. 

DARPA’s main tactic for executing its strategy is to constantly search 
worldwide for revolutionary high-payoff ideas and then sponsor projects 
bridging the gap between fundamental discoveries and the provision of 
new military capabilities. (DARPA, Strategic Mission) 

Mr. Kevin Wallace, one of the authors, has experienced first-hand the planning 
and insight that is required for a Program Manager to fulfill DARPA’s strategic vision. 
PMs amalgamate users, requirements developers, stakeholders, subject matter experts 
within industry and academia, and transition partners to develop innovative ideas into 
practical solutions. 

2. Dr. Karen Wood 

Dr. Wood joined DARPA in 2003, as Program Manager in the Tactical 
Technology Office. She joined the Strategic Technology Office in 2006. 

From 2002-2003, she supported SPO as an employee of ANSER working 
on chemical sensors for WMDs and large space structures with the ISAT 
program. While in TTO, she developed the Boomerang system, a rapid 
reaction, acoustic gunshot detection system, currently deployed in Iraq and 
Afghanistan. She also initiated the CROSSHAIRS, RPGnets and LASSO 
programs, along with numerous seedling efforts. From 1991 to 2001, Dr. 

Wood worked in various positions at NASA Langley Research Center in 
both the Advanced Materials Processing and Nondestructive Evaluation 
Sciences Branches. Her work included materials characterization and 
NDE of large space structures. (DARPA, STO) 
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B. BOOMERANG PROGRAM 


1. Background 

The Boomerang Program began in late 2003 as a result of an immediate need by 
U.S. troops in Iraq for a shooter detection system. There were several different types of 
detection systems already in use; however, there was a far less expensive system that 
could be more widely deployed and mounted on vehicles, as well as set up at a fixed 
location. The Boomerang was developed by DARPA as a Rapid Reaction Initiative. 
Under a Rapid Reaction Initiative, the goal is to field a solution within 60-90 days. 

2. Capability Gap 

The genesis of the Boomerang Program was U.S. Troops in Iraq experiencing a 
high number of causalities from small arms fire while driving in convoys. The 
adversaries were primarily using AK-47s while hiding along roadways at distances of 
less than 200 meters and would shoot in quick bursts from a concealed location (i.e., 
buildings, walls, etc.). Due to road noise often times the troops would not know they had 
been fired upon until they reached their destination and noticed bullet holes in their 
vehicle, a windshield was shattered by a bullet, or a casualty was incurred. The 
Boomerang program sought to remedy this problem by providing an affordable capability 
of detecting and located small arms fire. 

3. Boomerang I 

The prime contractor responsible for developing the Boomerang was BBN 
Technologies. BBN was involved in a previous DARPA effort aimed at developing 
counter sniper measures in the late 1990s, during which they developed Bullet Ears. 
Bullet Ears was a stationary system using two tetrahedral acoustic arrays separated by 50 
meters that processed the sound of a muzzle blast and the shockwave of the bullet to 
locate the shooter’s position (CNN, 1997). That impressive effort by BBN lead DARPA 
to seek them out for the Boomerang Program (Montiz, 2005). On November 17, 2003, 
BBN was awarded a contract for the development of the Boomerang system. This 
contract included the development and demonstration of a first article prototype within 30 
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days after contract award and included a go/no-go decision based on the demonstration to 
produce 50 systems. It also included six months of research and development covering 
all engineering and software development spirals, six optional lots of not to exceed 190 
systems per lot, and optional operations and support of the system. The initial 
Boomerang schedule is illustrated in Figure 8. 
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Figure 8. Boomerang Schedule 

The Initial Performance Objectives of the Boomerang were as follows: 

• Determines relative shooter direction to + 15 degrees within 1 second of 
shot 

• Shot Detection: 99% for bullet trajectory within 30 meters of sensor array 

• Operates at up to 60 mph on rough terrain or highways 

• Weapon Types: .20 cal to .50 cal (AK-47 = .30 cal) 

• Voice announcement and LED visual rosette display of shooter location 
(‘hours of a clock’) 
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Compact sensor array with full azimuth view 

Demonstrated performance in open fields and urban environments 

No false alarms due to wind noise, bumps, door slams, etc. 


Boomerang Major Components: 




UNIT 


Figure 9. HMMWV-mounted Boomerang System 
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Figure 10. Boomerang Block Diagram 

Boomerang I was designed, prototyped, tested, and the first lot of 50 units 
delivered in 66 days, only 6 days over the objective of 60 days. Out of first lot, 40 units 
were provided to the Army and 10 units were provided to the Marines. These units were 
deployed throughout Iraq. Users gathered data related to system performance and issues 
and communicated them to BBN and DARPA. 

4. Boomerang II 

As a result of areas for improvement identified by troops using the Boomerang I, 
upgrades were made to the system resulting in the second generation Boomerang or 
Boomerang II. 

The following issues were identified from the deployed Boomerang I systems: 

• High false alarm rate 

- EMI was determined to be the cause. 
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Fix: Shielding throughout system; work with EMI experts to mitigate 
problems 


• Overheated wiring 

- Power conditioners for 50 units were supplied by various vendors world¬ 
wide. Several were substandard. 

- Fix: Removed power conditioners and use jumpers for initial systems. 
Current systems does not require them 

• Rubber hub isolation mount failures 

- Batch specific failures due to rubber variations. Drilled holes and used 
doubled-stranded wire to support hub. 

- Fix: Hub redesign 

• Speaker bulky and falls off 

- Fix: Speaker integrated into display 

As a result, the identification of these issues and areas for improvement 
Boomerang IPs Performance Objectives were set as follows: 

• Determines relative shooter direction to + 5 degrees within 1 second of shot 

• Shot Detection: 99% for bullet trajectory within 50 meters of sensor array 

• Operates at up to 60 mph on rough terrain or highways or as a stationary 
stand- alone unit 

• Weapon Types: .20 cal to .50 cal (AK-47 = .30 cal) 

• Compact sensor array with full azimuth view 

• Voice announcement and FED visual rosette display of shooter location 
(‘hours of a clock’) 

• New display shows range and elevation along with 10-digit grid coordinate of 
vehicle 

• No false alarms due to wind noise, bumps, door slams, etc. 
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• No false alarms due to outgoing return fire 

• System cost is < $10K 

In June 2004, lot 2 was ordered consisting of 50 Boomerang IIs. All of these 
units were provided to the Army and were deployed throughout Iraq. In order to address 
the identified issues with Boomerang I and meet the updated performance objectives 
Boomerang II improvements updates included a redesigned sensor array/hub, a 
redesigned display, a simplified hardware layout, and improved algorithms. In this 
upgraded configuration Boomerang II met all of its performance objectives. Figure 12 
illustrates Boomerang II improvements. 
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Figure 11. Boomerang II Improvements 
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In total, over 500 Boomerang II units were eventually ordered and fielded. 
DARPA involvement in the Boomerang system officially in ended in January 2006 when 
its contract with BBN expired. User feedback on Boomerang IIs deployed in Iraq 
eventually led to the Boomerang III. 

5. Conclusion 

Boomerang is a great example of technology being quickly transitioned out of the 
S&T community and into the hands of the Warfighter to meet an immediate need. 
Boomerang I was designed, prototyped, tested, and the first lot of 50 units delivered in 66 
days. Once fielded, DARPA and BBN worked closely with users to identify areas of 
improvement to the system that resulted in Boomerang II. After DARPA’s involvement 
with Boomerang ended, additional improvements to the system were made eventually 
resulting in the current system configuration. Boomerang III. Boomerang III is a 
commercial item available to U.S. military, law enforcement, and other authorized 
entities through the GSA schedule. 
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VI. SUMMARY AND RECOMMENDATIONS 


A. SUMMARY 

This Joint Applied Project has examined the current DoD technology transition 
practices in the context of organization, policy, and metrics along with a case study 
demonstrating a very successful technology transition. A significant hurdle in technology 
transition within DoD has been “bridging the gap” between the S&T and acquisition 
communities to successfully transition promising technologies from the lab into the hands 
of the Warfighter. Recommendations are provided below on ways DoD can improve the 
success of its technology transition capabilities. 

B. RECOMMENDATIONS 
Recommendation #1 

Recommend DoD develop and implement an alternate mechanism to augment the 
JCIDS process to rapidly field promising technologies to meet immediate and short-term 
Warfighter needs. This would entail developing a more streamlined requirements 
approval process for immediate and short-term needs where the S&T community has 
identified potential technologies currently in development that have a high likelihood of 
meeting those needs and being matured, developed, and fielded. In addition, due to the 
difficulty obtaining funding to transition technologies that are not tied to a JROC vetted 
requirement, the alternate process would need to address a streamlined funding 
mechanism to allow the S&T and EMD community to rapidly secure funding to execute 
these projects. 

Currently within the U.S. Army, organizations are implementing a mechanism 
similar to what the authors are suggesting. The Rapid Equipping Eorce (REE) was 
established 

to equip operational commanders with commercial-off-the-shelf and 
government-off-the-shelf solutions to increase effectiveness and reduce 
risk, [and] insert future force technologies and surrogates to validate 
concepts and speed capabilities to the Soldiers, (www.ref.army.mil, n.p.). 
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The REF serves as a solutions catalyst, canvassing the military, industry, 

academia, and the science community for existing and emerging 

technologies, (www.ref.army.mil, n.p.) 

Also within the U.S. Army, Prototype Integration Facilities (PIFs) have been 
established to rapidly deliver solutions to the Warfighter. These solutions can be organic, 
modified commercial-off-the-shelf, or a combination of the two. Mr. Kevin Wallace, one 
of the authors, is a Branch Chief within one of the Army PIFs and has worked closely 
with the REF on several efforts. He has experienced first-hand several successful 
technology transitions; however, the process to bridge the gap between the S&T and 
EMD communities remains to be a somewhat arduous task. The concept behind the REF 
and PIFs is a step in the right direction and similar mechanisms should be proliferated 
across the joint services. 

Recommendation #2 

Recommend DoD implements policy requiring the industry proven practice of 
using relationship managers in the S&T and acquisition communities. These relationship 
managers would be located within the military lab communities and assigned to support a 
portfolio of technologies. They would function as liaisons between the labs, 
requirements developers, industry, academia, and the acquisition community. They 
would be responsible for interfacing with these communities to market technologies 
within their portfolio and transition those technologies (when appropriate) into the hands 
of the Warfighter. As illustrated by the Boomerang case study DARPA has had success 
using the relationship manager concept. The authors believe that the relationship 
manager concept would greatly aid the S&T and acquisition communities in successfully 
transitioning technology if implemented throughout DoD. 

Recommendation #3 

Recommend DoD adopt a gated technology development process consisting of a 
strategic planning phase, exploration phase, development phase, and a transition phase. 
This process is a common industry practice that has proven to be very successful in 
transitioning technology. In industry, successful transition begins with strong strategic 
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planning. During this planning process, market needs and technologies the company is 
developing (or potentially could develop to meet those needs) are identified. If 
recommendation two (2) were implemented by DoD along with recommendation three 
(3) the relationship managers would play a key role in this step by working with the users 
to determine their needs and the acquisition community to satisfy those needs. During 
this part in the process, the following questions must be answered: 

• What is the specific Warfighter need? This entails open bi-directional lines of 
communication with the end-user enforcing the bubble-up approach. 

• How soon is it needed? Is the need immediate, near-term or far-term, and is the 
timeline realistic. 

• Are there potential solutions that meet the timeframe? A thorough search of 
potential applicable solutions is a necessity. Are there technologies readily 
available that meet the user need, are of sufficient maturity, and can be 
transitioned within the timeframe required? 

• Can metrics be developed that lead to an operational user evaluation? TRLs, 
MRLs, and SRLs are critical to objectively measuring the maturity of a 
technology. These metrics as well as CTEs, Key Performance Parameters (KPP), 
and Measures of Effectiveness (MOE) help determine the extent to which the 
technology is appropriate for the solution and guide the development of 
downstream user evaluation criteria. Additionally, one could establish technical 
performance benchmarks which can be monitored throughout the maturation 
process to ensure the technology is remaining focused on the solution. 

• What is the schedule that will drive the program? What are the factors driving 
the schedule? Both essential questions in detailing the path forward. 

After strategic planning a gated process is followed going through exploration, 
development, and technology transition phases, with reviews taking place in between. 
When a given technology is ready for transition the technology transition community and 
the product development community work together to gradually shift management and 
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funding responsibilities from one to the other. The authors believe that implementing 
policy within DoD in line with these practices would aid in the future success of DoD’s 
transition capabilities. 

Recommendation #4 

Recommend the continual implementation of TRLs and MRLs as maturity 
metrics with the expanded usage of SRLs to encompass the bigger picture. The ability to 
discern the readiness of a technology in an objective manner is critical to the technology 
transition process. A system in development is only as mature as its least mature 
component. TRLs, MRLs, and SRLs aid developers in identifying the areas of risk. 
Once identified, strategic plans can be formulated to ensure all technologies are of 
sufficient maturity to transition to the next phase. 

The maturation of a technology is only a portion of the decision to transition to 
the next phase of development. Metrics such as MOEs, Technical Performance Measures 
(TPMs), and Design-dependent Parameters (DDPs) should be established and monitored 
to ensure the technology will ultimately fulfill the established need. MOEs, TPMs, DDPs 
and other Systems Engineering metrics demarcate the technology as it develops through 
the TRE, MRE, and SRE maturity levels. 

C. CONCLUSION 

In conclusion, this Joint Applied Project has examined the current DoD 
technology transition practices in the context of organization, policy, and metrics. Within 
DoD their continues to be a gap between the S&T and acquisition communities that must 
be more successfully bridged in order to transition technologies out of the lab and into the 
hands of the Warfighter. In order to do this, the authors believe that an alternate 
mechanism to the JCIDS process should be developed to rapidly field promising 
technologies to meet immediate and short-term Warfighter needs, the use of relationship 
managers within the military lab communities should be required, a gated technology 
development process based on common industry practice should be adopted, and the use 
of metrics measuring not only the maturity of the technology but also the extent to which 
it meets the users need should be expanded. 
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